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FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI),  as  part  of  its 
Training  for  Interactive  Distributed  Environments  work  package,  is  investigating  the  use  of 
training  technology  that  is  effective,  affordable,  and  distributable.  ARI  seeks  to  provide  the 
Army  with  guidance  on  how  game-based  training  tools  can  be  used  for  military  training. 

The  current  research  effort  was  a  follow-up  investigation  to  assess  how  trainee 
characteristics  impact  training  outcomes  in  a  game-based  training  environment.  The  game  used 
for  this  research  was  America ‘s  Army.  America ’s  Army  was  developed  by  the  Office  of 
Economic  and  Manpower  Analysis  to  serve  as  an  interactive  tool  for  providing  potential  recruits 
with  information  regarding  U.S.  Army  opportunities.  In  March  2005,  America ’s  Army  was  used 
during  a  four-day  training  exercise  at  the  U.  S.  Military  Academy.  This  game  was  used  to 
further  develop  cadet  tactics  skills  and  was  chosen  because  of  its  ability  to  allow  for  the  virtual 
simulation  of  small  team  maneuvers. 

An  initial  summary  of  this  research  was  briefed  to  representatives  from  the  Training  and 
Doctrine  Command — Training  Development  and  Analysis  Directorate;  the  Research, 
Development,  Experimentation  Command/Simulation  and  Training  Technology  Center;  the 
Army  Research  Lab  -  Human  Research  &  Engineering  Directorate;  and  the  Institute  for  Creative 
Technologies  in  September  2005.  Portions  of  the  findings  from  this  research  were  also  presented 
at  the  2U*  Annual  Conference  of  the  Society  for  Industrial/Organizational  Psychology  in  May 
2006,  the  Serious  Games  Summit  in  October  2005,  and  I/ITSEC  in  December  2005. 


STANLEY  M^ALPIN 
Acting  Technical  Director 
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VIDEOGAME-BASED  TRAINING  SUCCESS:  THE  IMPACT  OF  TRAINEE 
CHARACTERISTICS  -  YEAR  2 

EXECUTIVE  SUMMARY 


Research  Requirement: 

Personal  Computer  (PC)-based  videogames  are  emerging  as  an  increasingly  popular 
training  tool  in  the  U.  S.  Army.  The  present  research  represents  a  follow-up  investigation  to 
Orvis,  Orvis,  Belanich,  and  Mullin  (2005)  with  regards  to  the  impact  of  trainee  characteristics  in 
videogame-based  training  environments.  Specifically,  this  follow-up  research  examines  prior 
videogame  experience,  videogame  self-efficacy,  and  goal  orientation  as  antecedents  that 
maximize  trainee  motivation,  as  well  as  other  learner  choices  and  outcomes,  in  PC  game-based 
training. 

Procedure: 

In  this  research,  participants  played  America 's  Army,  a  first-person-perspective 
videogame  as  part  of  a  tactics  training  exercise.  The  videogame  began  with  a  single-player 
section  to  introduce  game-specific  tasks,  followed  by  a  multi-player  section  where  participants 
formed  small  teams  to  conduct  several  collaborative  missions.  Prior  to  and  after  the  training 
exercise,  cadets  were  asked  to  complete  online  questionnaires.  364  cadets  completed  the 
pretraining  questionnaire,  which  assessed  trainee  characteristics,  such  as  videogame  experience 
and  goal  orientation,  and  pretraining  motivation.  80  of  the  364  cadets  completed  the  posttraining 
questionnaire,  which  assessed  several  learner  outcomes/choices,  such  as  trainee  satisfaction  with 
the  training  experience  and  time  spent  engaging  in  the  training  game. 

Findings: 

Results  demonstrate  that  these  trainee  characteristics,  as  a  set,  had  a  positive  impact  on 
trainee  motivation,  trainee  satisfaction  with  the  training  experience,  ease  in  using  the  training 
game  interface,  team  cohesion,  metacognitive  strategies  utilized  during  training,  and  time  spent 
engaging  in  the  training  game.  Further,  consistent  with  Year  1  of  this  research  initiative,  the 
present  results  suggest  that  prior  videogame  experience  positively  predicted  these  criteria  only 
when  trainees’  prior  experience  was  with  videogames  possessing  similar  characteristics  as  the 
current  training  game.  Results  also  parallel  our  Year  1  findings  in  that  60%  of  the  military 
participants  in  this  sample  reported  they  had  limited  or  no  prior  experience  playing  videogames. 

Utilization  and  Dissemination  of  Findings: 

The  results  of  this  research  provide  useful  information  to  training  game  developers  and 
instructors  using  videogames  as  training  tools.  This  research  extends  our  Year  1  findings,  as  well 
as  other  prior  work,  by  demonstrating  that  a  trainee’s  prior  experience,  self-efficacy,  and  goal 
orientation  beliefs  uniquely  contribute  to  understanding  when  game-based  training  should 
succeed  in  enhancing  trainee  motivation  and  other  important  learner  choices/outcomes. 
Modification  of  these  trainee  characteristics  prior  to  training  may  help  to  enhance  the 
effectiveness  of  PC  game-based  training. 
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VIDEOGAME-BASED  TRAINING  SUCCESS: 

THE  IMPACT  OF  TRAINEE  CHARACTERISTICS  -  YEAR  2 


INTRODUCTION 

PC-based  videogames  are  emerging  as  an  increasingly  popular  training  tool  in  the  U.  S. 
Army  (Herz  &  Macedonia,  2002).  One  rationale  for  using  game-based  simulations  for  training 
purposes  is  that  they  can  be  motivating  to  use,  and  it  is  easier  to  train  motivated  learners  than 
non-motivated  learners  (Malone,  1981;  Prensky,  2001).  Further,  research  demonstrates  that 
increased  trainee  motivation,  and  hence  time  and  effort  devoted  to  training,  subsequently 
improves  learning  outcomes  such  as  knowledge/skill  acquisition  and  retention  (Colquitt,  LePine, 
&  Noe,  2000;  Mathieu,  Tannenbaum,  &  Salas,  1992;  Tannenbaum  &  Yukl,  1992). 

Unfortunately,  research  also  suggests  that  in  self-directed,  technology-delivered  training 
environments  (such  as  e-leaming  and  game-based  training  environments)  trainees  with  low 
motivation  sometimes  terminate  training  before  mastering  the  intended  training  objectives  (Bell 
&  Kozlowski,  2002;  Steinberg,  1989;  Tennyson,  1980).  Such  trainees  are  clearly  at  a 
disadvantage.  Consequently,  determining  how  to  motivate  trainees  to  continue  engaging  in 
game-based  learning  environments  is  of  great  concern  to  instructors  and  trainers. 

Research  on  training  games  has  primarily  focused  on  videogame  features  (e.g.,  the 
training  game’s  level  of  challenge,  realism,  and  interactivity)  that  may  influence  trainee 
motivation  and  other  training  outcomes  of  game-based  training  environments.  Indeed,  such 
game  features  have  been  found  to  enhance  a  trainee’s  knowledge  acquisition  and  motivation  to 
learn  (Belanich,  Sibley,  &  Orvis,  2004;  Corbeil,  1999;  Garris,  Ahlers,  &  Driskell,  2002).  This 
prior  research  has  enhanced  our  understanding  of  how  to  design  the  actual  training  game  to 
improve  its  effectiveness.  Yet,  to  date,  little  research  has  examined  individual  characteristics  of 
the  trainee  that  may  facilitate  or  impede  trainee  motivation.  The  present  research  focuses  on 
determining  individual  characteristics  that  maximize  trainee  motivation  in  game-based  training 
environments. 

According  to  social  cognitive  theory,  individuals’  cognitions  and  behaviors  during  a 
learning  experience  are  influenced  by  two  motivational  constructs:  a)  self-efficacy  -  personal 
perceptions  regarding  one’s  ability  to  accomplish  a  given  task  and  b)  goals  -  one’s  reasons  for 
engaging  in  the  task  (Bandura,  1986;  Pintrich  &  Schunk,  1996).  Traditional  training  and 
e-leaming  research  has  shown  that  self-efficacy  and  goal  orientation  are  related  to  trainee 
motivation,  time  on  task,  and  other  training  outcomes  (e.g..  Brown,  2001 ;  Colquitt  et  al.,  2000; 
Fisher  &  Ford,  1998).  Prior  research  has  also  demonstrated  that  trainees’  previous  videogame 
experience  is  positively  related  to  time  on  task  in  game-based  training  environments  (Orvis  et  al., 
2005).  Thus,  this  paper  focuses  on  self-efficacy,  goal  orientation,  and  prior  videogame 
experience  as  antecedents  of  trainee  motivation,  as  well  as  other  learner  choices  and  outcomes. 
The  ability  to  identify  such  individual  attributes  that  lead  to  success  in  PC  game-based  training 
environments  will  help  to  better  prepare  Soldiers  for  training  and  will  lead  to  increased 
operational  capabilities. 
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Trainee  Motivation  and  Learner  Choices/Outcomes  Relevant  to  Game-Based  Training 

Pretraining  motivation,  also  referred  to  as  motivation  to  learn,  refleets  the  trainee’s  desire 
to  learn  the  eontent  of  the  training  program  (Noe,  1986).  Pretraining  motivation  is  believed  to 
prepare  trainees  to  learn  by  heightening  their  reeeptiveness  to  new  ideas,  attention,  and  effortful 
behavior  during  the  training  experience  (e.g.,  Mathieu  et  ah,  1992).  Indeed,  research  has 
consistently  demonstrated  that  pretraining  motivation  predicts  both  cognitive  and  skill-based 
learning  outcomes  across  a  variety  of  settings  (Baldwin,  Magjuka,  &  Loher,  1991;  Colquitt  et  ah, 
2000;  Martocchio  &  Webster,  1992;  Noe  &  Schmitt,  1986;  Tannenbaum  &  Yukl,  1992). 

In  addition  to  pretraining  motivation,  the  current  research  examines  several  additional 
criteria  relevant  to  learning  in  videogame-based  training  environments.  Research  has  found  that 
individuals’  learner  choices  during  instruction  (i.e.,  their  choices  concerning  the  level  and  focus 
of  effort  to  expend)  influence  their  level  of  learning  in  technology-delivered  training 
environments  (Brown,  2001 ;  Schmidt  &  Ford,  2003).  Learner  choices  are  reflected  both  in  an 
individual’s  cognitions  (e.g.,  metacognitive  activity,  off-task  attention)  and  behaviors  (e.g.,  time 
on  task,  practice  behaviors)  during  the  learning  experience  (Brown,  2001).  Prior  research  also 
demonstrates  that  affect-based  outcomes  (e.g.,  training  satisfaction)  are  significantly  related  to 
learning  in  technology-delivered  training  environments  (e.g.,  Wasserman,  Orvis,  Fisher,  & 

Barry,  2002),  and  that  individuals’  performance  within  a  training  program  is  predictive  of 
knowledge/skill  transfer  (Ford  et  ah,  1998;  Kozlowski,  Gully,  Brown,  Salas,  Smith,  &  Nason, 
2001).  Thus,  in  addition  to  pretraining  motivation,  this  research  investigates  six  criteria  relevant 
to  videogame-based  learning:  the  learner  choices  of  (1)  metacognitive  activity  and  (2)  time  on 
task;  the  affect-based  outcomes  of  (3)  training  satisfaction,  (4)  ease  in  using  the  training  game’s 
user  interface,  and  (5)  perceived  cohesion  with  one’s  teammates  while  playing  the  game;  and  the 
skill-based  outcome  -  (6)  training  performance. 

Metacognitive  activity.  Metacognition  involves  planning,  monitoring,  and  revising  goal 
appropriate  behavior  (Brown,  Bransford,  Ferrara,  &  Campione,  1983).  Learners  who  engage  in 
greater  metacognitive  activity  during  training  should  learn  more  effectively  because  they  actively 
monitor  their  learning  progress,  self-evaluate  where  they  are  having  difficulties,  and  adjust  their 
behaviors  accordingly  to  address  these  difficulties  (Brown  et  ah,  1983;  Ford,  Smith,  Weissbein, 
Gully,  &  Salas,  1998).  Prior  research  supports  that  engagement  in  metacognitive  activity  during 
instruction  results  in  higher  knowledge  and  skill  acquisition  (Ford  et  ah,  1998;  Pintrich  & 
DeGroot,  1990;  Schmidt  &  Ford,  2003).  Metacognition  may  be  a  particularly  important  learner 
choice  in  game-based  training  because  this  type  of  learning  environment  typically  provides  little 
external  structure  or  feedback  regarding  the  most  effective  way  to  learn  while  progressing 
through  the  training  game. 

Time  on  task.  The  length  of  time  in  which  a  learner  is  engaged  in  learning  the  knowledge 
or  skills  to  be  mastered  during  instruction  has  consistently  been  demonstrated  to  be  an  important 
predictor  of  learning  in  both  educational  and  organizational  learning  contexts  (e.g.,  Bloom,  1974; 
Bloom,  1976;  Borg,  1980;  Brown,  2001;  Good  &  Beckerman,  1978).  For  example.  Brown 
(2001)  found  that  amount  of  time  spent  engaging  in  an  e-leaming  training  program  was 
positively  related  to  knowledge  acquisition. 
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Training  satisfaction.  Training  satisfaction  focuses  on  both  emotionally-based  opinions 
concerning  the  training  (e.g.,  the  trainee  liking  the  training)  and  reactions  regarding  the  utility  of 
the  training  (e.g.,  the  trainee  believing  the  training  enhanced  his/her  knowledge  or  skills). 
Trainees’  level  of  satisfaction  with  the  training  has  been  found  to  be  significantly  related  to 
learning  in  an  e-leaming  environment  (Wasserman  et  al.,  2002).  This  may  be  because  when 
trainees  are  more  satisfied  with  their  training  experiences,  they  are  likely  to  stay  engaged  for 
longer  periods  of  time  or  put  forth  greater  mental  effort  in  trying  to  learn  the  training  content, 
thus  resulting  in  greater  levels  of  learning. 

Ease  in  using  game  interface.  The  perception  of  ease  in  using  the  training  game  interface 
is  another  criterion  that  should  influence  the  level  of  engagement  in  a  videogame-based  training 
environment.  If  technology-delivered  learning  environments  are  frustrating  and  difficult  to  use, 
trainees  may  experience  decreased  motivation  and  not  fully  engage  in  the  instruction  (Park  & 
Tennyson,  1980;  Tennyson,  1980).  Difficulty  with  the  technology  or  interface  in  which  the 
instructional  content  is  delivered  has  been  cited  as  a  key  Ihistration  source  and  a  reason  for  low 
completion  rates  in  e-leaming  programs  (Frankola,  2001).  Moreover,  prior  research  has  found 
that  trainees’  perceptions  regarding  the  user  interface  of  an  e-leaming  program  were  positively 
related  to  their  satisfaction  with  the  overall  training;  which,  in  turn,  was  positively  related  to 
learning  (Wasserman  et  al.,  2002). 

Team  cohesion.  Many  training  videogames  are  collaborative  in  nature,  requiring  the 
interaction  and  cooperation  among  trainee  team  members  in  order  to  be  successful  in  the  game 
and  to  learn  the  instmctional  content.  Previous  collaborative  learning  research  has  found  that  the 
quality  of  intra-team  interactions  is  a  key  element  in  determining  the  extent  and  depth  of  learning 
in  such  environments  (Gilbert  &  Moore,  1998;  Northrup,  2001;  Shute,  Lajoie,  &  Gluck,  2000; 
Wagner,  1997).  Quality  team  interactions  originate,  in  part,  from  collaborative  team  states  such 
as  team  cohesion.  Indeed,  team  cohesion,  most  commonly  defined  as  members  liking  for  one 
another  (Evans  &  Jarvis,  1980)  and  the  extent  to  which  team  members  are  attracted  to  the  idea  of 
the  group  (Hogg,  1992),  has  been  found  to  be  directly  related  to  group  effectiveness  (Evans  & 
Dion,  1991;  Mullen,  Anthony,  Salas,  &  Driskell,  1993;  Mullen  &  Copper,  1994). 

Training  performance.  An  individual’s  performance  while  completing  a  training 
program  is  indicative  of  the  extent  to  which  he/she  is  acquiring  the  skills  or  knowledge  being 
taught  within  the  training.  Further,  prior  training  research  demonstrates  that  a  learner’s  training 
performance  is  positively  related  to  knowledge/skill  transfer  (Ford  et  al.,  1998;  Kozlowski  et  al., 
2001). 

Prior  Videogame  Experience 

To  date,  limited  gaming  research  has  attempted  to  identify  trainee  characteristics  that 
influence  learner  choices/outcomes  in  videogame-based  training  environments.  Prior  research 
has  found  that  an  individual’s  prior  videogame  experience  (i.e.,  frequency  of  game  use)  is 
predictive  of  his/her  future  performance  in  videogame-based  environments  (Gagnon,  1985; 
Young,  Broach,  &  Farmer,  1997).  Further,  the  Year  1  results  of  the  present  research  initiative 
demonstrate  that  a  trainee’s  level  of  prior  videogame  experience  predicts  his/her  learner  choices 
during  training  and  his/her  subsequent  affect-based  learning  outcomes;  prior  gaming  experience 
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significantly  predicted  a  trainee’s  time  spent  engaging  in  the  training  game,  satisfaction  with  the 
training  experience,  perceived  ease  in  using  the  training  game  interface,  and  team  cohesion 
(Orvis  et  al.,  2005). 

Moreover,  in  Year  1,  we  also  found  that  the  influence  of  prior  experience  on  these  learner 
choices/outcomes  was  dependent  on  the  specificity  of  a  trainee’s  prior  videogame  experience. 
Specifically,  prior  game  experiences  that  shared  similar  game  characteristics  to  the  given  training 
game  environment  (e.g.,  pace,  interface,  perspective)  were  more  likely  to  positively  predict  the 
learner  choices/outcomes.  Prior  experiences  with  irrelevant  games  (i.e.,  games  that  do  not  share 
similar  characteristics)  did  not  predict  these  choices/outcomes.  This  finding  has  also  been 
observed  with  respect  to  computer  experience,  in  that  specific  prior  computer  experiences  have 
been  found  to  be  differentially  predictive  of  learning  outcomes  in  computer-based  learning 
environments  (Salanova,  Grau,  Ciffe,  &  Llorens,  2000;  Woodrow,  1991). 

Consistent  with  our  Year  1  findings,  we  expected  in  the  current  research  that  trainees 
with  greater  levels  of  overall  videogame  experience  would  be  more  motivated  to  train  in  the 
given  videogame-based  training  environment  than  those  with  less  videogame  experience.  Prior 
videogame  experience  was  also  expected  to  positively  influence  learner  choices  during  training 
(i.e.,  metacognitive  activity  and  time  on  task),  subsequent  affeet-based  learning  outcomes  (i.e., 
training  satisfaction,  ease  in  using  game  interface,  and  perceived  team  cohesion),  and  training 
perfonnance.  Further,  we  expected  that  only  experience  with  relevant  videogames  (i.e.,  game 
types  that  share  similar  game  characteristics  to  the  given  training  game  environment)  would 
positively  influence  these  training  criteria. 

Videogame  Self-Efficacy 

Self-efficacy  is  a  judgment  of  one’s  capability  to  successfully  perform  a  specific  task 
(Bandura,  1977).  Self-efficacy  is  a  domain  specific  construct  in  that  it  varies  across  different 
types  of  tasks  and  situational  contexts  (Bandura,  1977).  As  such,  of  particular  relevance  to 
videogame-based  training  environments  is  videogame  self  efficacy  -  a  judgment  of  one’s 
capability  to  successfully  play  videogames. 

Trainee  self-efficacy  beliefs  have  been  found  to  be  an  important  predictor  of  pretraining 
motivation  (Colquitt  et  al.,  2000),  amount  of  time  spent  practicing  new  skills  (Bouffard- 
Bouchard,  1990),  trainee  reactions  to  a  training  program  (Mathieu,  Martineau,  &  Tannenbaum, 
1993),  and  learning  and  performance  (Bandura  &  Cervone,  1986;  Colquitt  et  al.,  2000;  Gist, 
Schwoerer,  &  Rosen,  1989;  Martocchio  &  Judge,  1997;  Martocchio  &  Webster,  1992)  in  various 
training  contexts,  including  technology-delivered  training  eontexts.  Year  1  of  this  research 
initiative  also  demonstrated  the  positive  impact  of  self-efficacy  in  videogame-based  training 
environments.  Specifically,  we  examined  trainees’  levels  of  computer  self-efficacy  and  found 
that  this  type  of  self-efficacy  provided  incremental  validity  over  prior  videogame  experience  in 
the  prediction  of  time  spent  engaging  in  the  training,  perceived  ease  in  using  the  training 
interfaee,  and  team  cohesion  (Orvis  et  al.,  2005). 

In  general,  prior  research  suggests  that  individuals  with  high  self-efficacy  tend  to  exert 
greater  mental  effort  and  persistence  while  completing  a  training  program;  and  thus,  experience 
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more  positive  cognitive,  skill,  and  affect-based  learning  outcomes.  Thus,  consistent  with  this 
prior  work,  we  expected  in  the  current  research  that  trainees  with  greater  videogame  self-efficacy 
would  be  more  motivated  to  train  in  a  videogame-based  training  environment  than  those  with 
lower  videogame  self-efficacy.  Further,  trainees’  level  of  videogame  self-efficacy  was  expected 
to  positively  influence  their  learner  choices  during  training,  subsequent  affect-based  learning 
outcomes,  and  training  performance. 

Goal  Orientation 

Goals  are  widely  recognized  as  being  central  to  the  understanding  of  motivated  behavior. 
Dispositional  goal  orientation  theory  suggests  that  individuals  adopt  distinct  outlooks  or  mental 
frameworks  regarding  learning  and  achievement  contexts  (Brett  &  Vandewalle,  1999).  These 
differing  frameworks  influence  individuals’  reasons  for  engaging  in  leaming/achievement  tasks, 
beliefs  regarding  causes  of  success,  and  preferences  regarding  task  difficulty  (Dweck,  1986; 

Farr,  Hofmann,  &  Ringenbach,  1993).  Thus,  a  trainees’  goal  orientation  should  influence  his/her 
cognitions  and  behaviors  during  a  learning  experience.  To  the  authors’  knowledge,  goal 
orientation  has  not  been  examined  within  the  context  of  videogame-based  training  environments. 

Learning  goal  orientation.  Learning  goal  orientation  is  a  dedication  to  developing 
competence  by  acquiring  new  skills,  mastering  novel  situations,  and  learning  from  experience 
(Dweck,  1986;  Dweck  &  Leggett,  1988;  Vandewalle,  1997).  Learning-oriented  learners  view 
ability  and  skills  as  malleable.  They  tend  to  seek  out  novel  or  challenging  situations  in  order  to 
increase  their  competence  on  a  given  task  (Dweck,  1986;  Kozlowski  et  ah,  2001).  As  such,  these 
individuals  perceive  training  as  an  opportunity  to  learn,  and  they  believe  demonstrating  effort 
and  persistence,  even  in  the  face  of  difficulties,  is  worthwhile  for  inereasing  one’s  competenee. 

Prior  research  conducted  on  classroom  and  technology-delivered  training  has 
demonstrated  that  Learning  goal  orientation  is  positively  related  to  learning  (Fisher  &  Ford, 

1998)  and  pretraining  motivation  (Colquitt  &  Simmering,  1998).  Learning  goal  orientation  has 
also  been  associated  with  positive  effects  on  learner  choices  during  training.  Trainees  who 
approach  learning  environments  with  the  purpose  of  mastering  new  knowledge/skills  engaged  in 
greater  metacognitive  activity  (Ford  et  ah,  1998;  Schmidt  &  Ford,  2003),  reported  decreases  in 
their  off-task  attention  (Brown,  2001),  and  demonstrated  greater  effort  during  training  (Ames  & 
Archer,  1988;  Fisher  &  Ford,  1998).  Further,  prior  research  has  demonstrated  beneficial  effects 
of  Learning  goal  orientation  on  an  individual’s  affective  reactions  (e.g.,  satisfaction  with 
performance;  Jagaeinski  &  Nicholls,  1984;  Treasure  &  Roberts,  1994). 

Accordingly,  we  expected  that  trainees  with  high  Learning  goal  orientation  would  be 
more  motivated  to  train  in  a  videogame-based  training  environment  than  those  with  lower  levels 
of  Learning  goal  orientation.  Further,  we  expected  Learning  goal  orientation  to  positively 
influence  learner  choices  of  metacognitive  activity  and  time  on  task,  the  affect-based  learning 
outcome  of  training  satisfaction,  and  training  performance. 

Performance  goal  orientation.  Individuals  with  high  Performance  goal  orientation 
believe  their  ability  and  skill  levels  are  stable  and  unlikely  to  change.  Performance-oriented 
learners  focus  on  demonstrating  and  validating  their  competence  by  seeking  good  performance 
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evaluations  and  avoiding  negative  ones  (Dweck,  1986).  These  learners  prefer  learning 
environments  that  are  familiar  and  do  not  require  much  effort  to  master  (i.e.,  learning 
environments  that  ensure  positive  evaluations  of  their  capabilities)  because  their  concern  with 
competence  is  more  about  superficial  demonstration  than  substantive  development  (Campbell  & 
Kuncel,  2001;  Vandewalle,  Cron,  &  Slocum,  2001).  Accordingly,  the  desire  for  a  positive 
evaluation  regarding  one’s  competency  does  not  necessarily  correspond  with  engaging  in 
cognitions  and  behaviors  needed  to  actually  develop  competence.  In  fact,  performance-oriented 
learners  may  withdraw  effort  or  avoid  difficult  learning  tasks  all  together  (Ames,  1992;  Duda  & 
Nicholls,  1992). 

Initial  research  results  for  the  influence  of  Performance  goal  orientation  on 
leaming/performance  outcomes  have  tended  to  be  inconsistent,  sometimes  exhibiting  effects  but 
often  failing  to  do  so  (Beaubien  &  Payne,  1999).  Much  of  this  initial  work  was  based  on  a  two- 
factor  model  of  goal  orientation  -  Learning  and  Performance  goal  orientation  (e.g.,  Button, 
Mathieu,  &  Zajac,  1996;  Fisher  &  Ford,  1998;  Ford  et  ah,  1998).  However,  several  researchers 
have  suggested  that  goal  orientation  is  better  conceptualized  as  a  three-factor  construct,  with 
Performance  goal  orientation  consisting  of  two  separate  dimensions:  Performance  Avoid  and 
Performance  Prove  (Vandewalle,  1997;  Vandewalle  et  ah,  2001).  Elliot’s  (1994)  meta-analysis, 
as  well  as  other  studies  (e.g.,  VandeWalle  &  Cummings,  1997),  has  provided  evidence  that  the 
Performance  Avoid  and  Performance  Prove  dimensions  of  Performance  goal  orientation  have 
differential  relationships  with  various  outcome  variables.  Accordingly,  we  will  examine  the 
independent  effects  of  these  two  dimensions  on  the  learner  choices/outcomes,  in  addition  to  the 
effects  of  Learning  goal  orientation. 

Perfonnance  Avoid  goal  orientation  focuses  on  avoiding  negation  of  one’s  competence, 
demonstrations  of  low  ability/skill  levels,  and  negative  evaluations  from  others  (Brett  & 
Vandewalle,  1999;  Vandewalle,  1997).  Prior  research  has  generally  demonstrated  that 
Performance  Avoid  goal  orientation  is  associated  with  negative  effects  on  learner  choices  during 
training  and  subsequent  learning  outcomes  (Elliot  &  Harackiewicz,  1996;  Ford  et  ah,  1998; 
Schmidt  &  Ford,  2003;  Vandewalle  et  ah,  2001).  For  instance,  Schmidt  and  Ford  (2003)  found 
that  trainees  with  a  higher  Performance  Avoid  goal  orientation  engaged  in  less  metacognitive 
activity  during  training.  Metacognition  involves  actively  monitoring  for  and  evaluating/ 
determining  content  areas  in  which  one  is  having  personal  difficulties  mastering.  Individuals 
with  high  Perfonnance  Avoid  goal  orientation  seek  to  avoid  evaluations  concerning  personal 
areas  in  need  of  improvement,  regardless  of  whether  the  evaluation  is  other-  or  self-generated; 
thus,  they  tend  to  avoid  engagement  in  metacognitive  activity.  As  another  example,  Elliot  and 
Harackiewicz  (1996)  demonstrated  that  trainees  in  a  high  Performance  Avoid  goal  orientation 
training  group  spent  less  time  on  task  as  compared  to  those  in  the  high  Learning  goal  orientation 
or  Perfonnance  Prove  orientation  induced  training  groups. 

In  contrast,  Performance  Prove  goal  orientation  focuses  on  demonstrating  or  proving 
one’s  competence  by  outperforming  others  and  gaining  favorable  evaluations  from  others  (Brett 
&  Vandewalle,  1999;  Vandewalle,  1997).  The  central  difference  between  Performance  Prove 
and  Performance  Avoid  concerns  whether  one  is  primarily  driven  to  outperform  others  or  to 
avoid  failure.  Research  is  still  inconclusive  about  the  role  of  Performance  Prove  goal 
orientation.  In  general,  Perfonnance  Prove  goal  orientation  has  failed  to  demonstrate  a 
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consistent  positive  or  negative  relationship  with  any  given  learning  eriterion,  ineluding 
metacognitive  aetivity,  goal  setting,  post-training  self-efficacy,  knowledge  acquisition,  or 
skill-based  learning  (e.g.,  Sehmidt  &  Ford,  2003;  Vandewalle  et  al.,  2001). 

Based  on  this  prior  work,  we  expected  that  trainees  with  high  Performance  Avoid  goal 
orientation  would  be  less  motivated  to  train  in  a  videogame-based  training  environment  than 
those  with  lower  levels  of  Performance  Avoid  goal  orientation.  Further,  in  general,  we  expected 
Performance  Avoid  goal  orientation  to  negatively  influence  learner  choices  during  training, 
affect-based  learning  outcomes,  and  training  performance.  We  did  not  expect  to  find  any 
significant  relationships  between  Performance  Prove  goal  orientation  and  these  training  criteria. 

METHOD 


Participants 

Participants  were  first-year  U.  S.  Military  Academy  cadets  who  took  part  in  a  game- 
based  tactics  training  exercise.  The  mean  age  of  participants  was  1 8.89  years  (SD  =  1 .26  years). 
Prior  to  and  after  the  four-day  training  exercise,  cadets  were  asked  to  complete  online 
questionnaires  for  the  current  research.  Completion  of  the  pretraining  and  posttraining  research 
questionnaires,  while  encouraged  by  instructors,  was  voluntary.  The  pretraining  questionnaire 
assessed  trainees’  individual  characteristics  and  pretraining  motivation  for  the  game-based 
training  exercise;  of  the  approximately  1000  cadets  who  participated  in  the  training  exercise,  364 
cadets  completed  the  pretraining  questionnaire.  The  posttraining  questionnaire  assessed  learner 
choices  and  outcomes,  ineluding  metacognitive  activity,  time  on  task,  training  satisfaction, 
perceived  ease  in  using  the  training  game  interfaee,  perceived  team  cohesion,  and  training 
performance;  80  of  the  364  cadets  completed  the  posttraining  questionnaire. 

Game 


The  videogame  used  in  the  training  exereise  was  America 's  Army,  an  online  PC-based, 
first-person-perspective  game  with  both  single-player  and  multi-player  seetions.  America 's 
Army,  created  by  the  Office  of  Economie  and  Manpower  Analysis  at  the  U.  S.  Military 
Aeademy,  was  originally  developed  to  serve  as  a  recruiting  tool  in  order  to  inform  potential 
recruits  about  what  to  expect  during  basic  training  and  about  Army  core  values,  history,  and 
Army  background.  The  distribution  of  America ’s  Army  has  been  extensive,  with  over  four 
million  registered  players  (Petermeyer,  2004).  This  game  was  ehosen  for  this  exercise  because 
of  its  ability  to  simulate  small  team  environments  that  require  decision  making  and  collaboration 
skills.  Additional  information  regarding  America ’s  Army  is  available  at 
http://www.  americasarmy.  com . 

Procedure 

Prior  to  the  start  of  the  tactics  training  exercise,  the  instructor  provided  the  Web  address 
of  the  questionnaires  and  informed  the  cadets  of  the  opportunity  to  partieipate  in  this  research. 
Cadets  were  provided  with  an  introduction  to  the  questionnaires  and  promised  confidentially  of 
their  responses.  Interested  cadets  completed  the  questionnaires  on  their  own  time. 
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During  the  four-day  training  exercise,  cadets  played  the  computer  game  online  during 
their  own  time,  at  a  location  of  their  preference.  First,  the  cadets  completed  a  “basic  training” 
single-player  section,  where  they  learned  how  to  play  the  game.  This  section  contained  four 
segments:  a)  marksmanship  training,  b)  an  obstacle  course,  c)  weapons  familiarization,  and  d)  a 
MOUT  (military  operations  in  urban  terrain)  training  mission.  In  the  marksmanship  segment, 
cadets  practiced  using  a  computer  mouse  to  shoot  a  rifle.  Practice  and  qualification  rounds  were 
repeated  until  the  cadet  qualified  with  his/her  weapon  (at  least  23  out  of  40  targets  with  40 
rounds  of  ammunition).  In  the  second  segment,  the  obstacle  course,  cadets  completed  a  course 
which  includes  obstacles  such  as  climbing  over  a  wall,  running  over  a  balance  beam,  and  low 
crawling  under  barbwire.  Cadets  repeated  the  obstacle  course  until  they  bettered  the  time 
requirement  of  90  seconds.  In  the  weapons  familiarization  segment,  cadets  practiced  using  a 
computer  mouse  to  operate  four  different  weapons  including  a  machine  gun,  rifle  with  a  grenade 
launcher,  fragmentation  grenade,  and  smoke  grenade.  Cadets  were  not  required  to  meet 
qualification  standards  with  these  weapons.  In  the  final  segment,  MOUT  training,  cadets 
navigated  through  a  building  and  several  tunnels  while  being  introduced  to  and  practicing  basic 
“rules  of  engagement”  (i.e.,  shooting  at  stationary  silhouettes  of  “hostile”  targets  while  not  firing 
at  the  silhouettes  of  “noncombatant”  targets).  This  section  required  the  use  of  shooting  and 
movement  skills  introduced  in  the  prior  three  sections.  Cadets  repeated  the  MOUT  training 
exercise  until  they  achieved  a  target  score  (based  on  a  combination  of  shooting  hostile  targets, 
not  shooting  noncombatant  targets,  and  completing  the  exercise  quickly). 

Once  the  basic  training  section  had  been  completed,  cadets  were  eligible  to  play  the 
multi-player  section  of  the  game.  In  the  multi-player  section,  cadets  were  placed  into  small 
teams  and  engaged  in  collaborative  missions.  No  minimum  or  maximum  limit  of  missions  was 
set.  Team  membership  was  not  fixed;  instead,  membership  could  vary  across  collaborative 
missions  depending  on  which  cadets  were  currently  online  engaging  in  the  game.  In  the  mission, 
a  team’s  goal  was  either  to  attack  or  to  defend  a  radio  tower.  Regardless  of  the  team’s  goal 
within  a  given  mission,  cadets  took  the  perspective  of  a  U.S.  Soldier,  while  the  opposing  team 
was  depicted  as  the  enemy. 

The  multi-player  section  of  America ’s  Army  represents  a  distributed,  online  environment 
because  all  team  members  engage  in  the  same  mission  during  “real”  time.  However,  each  team 
member  plays  the  game  on  an  individual  computer,  in  a  different  physical  location.  Team 
members  interact  in  tenus  of  observing  each  other’s  actions  during  the  “real  time”  mission  and 
via  written  communication  using  an  online  chat  feature  built  into  the  game  interface. 

After  the  completion  of  the  training  exercise,  the  posttraining  questionnaire  was  posted 
online.  Cadets  who  completed  the  pretraining  questionnaire  were  provided  with  the  opportunity 
to  receive  a  reminder  email  for  this  posttraining  questionnaire.  Interested  cadets  provided  the 
researchers  with  an  email  address  to  which  a  reminder  was  sent. 

Measures  Prior  to  Training 

Overall  videogame  experience.  Overall  game  experience  was  assessed  using  one  item 
adopted  from  Orvis  et  al.  (2005),  “In  the  past  year,  how  frequently  have  you  played 
videogames?”  Possible  responses  ranged  from  0  (never)  to  5  (very  frequently). 
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Specific  videogame  experience.  Prior  game  experience  with  specific  games  was  assessed 
using  a  nine-item  scale  adopted  from  Orvis  et  al.  (2005).  Cadets  were  asked  to  note  how 
frequently  they  played  a  specific  type  of  videogame,  using  a  6-point  Likert  scale  ranging  from  0 
(never)  to  5  (very  frequently).  The  nine  types  of  specific  game  experience  assessed  were:  a) 
first-person-perspective  (e.g.,  Battlefield  1942,  James  Bond  007,  Medal  of  Honor);  b)  flight 
simulators  (e.g.,  Microsoft  Flight  Simulator,  Lock  On:  Modem  Air  Combat);  c)  massively  multi¬ 
player  online  games  (e.g.,  EverQuest,  Ultima  Online);  d)  sports/racing  (e.g..  Madden  NFL  2005; 
Tony  Hawk  Underground);  e)  military  command/strategy  (e.g.,  Rome,  Axis  &  Allies,  Rise  of 
Nations);  f)  fighting  (e.g.,  WWE  Smackdown,  Mortal  Kombat  Deception);  g)  life/business 
simulations  (e.g.,  The  Sims,  Tycoon);  h)  fantasy/adventure  (e.g.,  Myst  IV,  Revelation,  Syberia 
2);  and  i)  puzzles/card  games/board  games  (e.g..  Minesweeper,  Solitaire,  Chess).  In  addition, 
participants  were  asked  to  indicate  the  extent  to  which  they  had  previously  played  America 's 
Army. 


Videogame  self-efficacy.  Videogame  self-efficacy  was  assessed  using  two  items 
consisting  of  “I  am  certain  I  will  be  successful  at  most  videogames  I  try  to  play”  and  “I  am 
confident  in  my  ability  to  play  videogames.”  Possible  responses  ranged  from  1  (strongly 
disagree)  to  5  (strongly  agree).  The  coefficient  alpha  for  this  scale  was  .92. 

Goal  orientation.  Goal  orientation  was  assessed  using  a  13-item  scale  adapted  from 
Vandewalle  (1997).  Five  items  assessed  a  trainee’s  Learning  goal  orientation.  An  example  item 
is  “I  often  look  for  opportunities  to  develop  new  knowledge  and  skills.”  Performance  Avoid 
goal  orientation  was  assessed  with  four  items;  an  example  item  is  “I  would  avoid  taking  on  a 
new  task  if  there  was  a  chance  that  I  would  appear  rather  incompetent  to  others.”  Performance 
Prove  goal  orientation  was  assessed  with  four  items;  an  example  item  is  “I  prefer  to  work  on 
tasks/assignments  where  I  can  prove  my  ability  to  others.”  Possible  responses  ranged  from  1 
(strongly  disagree)  to  5  (strongly  agree).  The  coefficient  alpha  for  Learning  goal  orientation. 
Performance  Avoid  goal  orientation,  and  Performance  Prove  goal  orientation  were  .85,  .82,  and 
.78,  respectively.  All  items  of  these  scales  are  provided  in  Appendix  A. 

Pretraining  motivation.  Pretraining  motivation  for  the  game-based  training  program  was 
assessed  using  a  five-item  scale  adapted  from  Noe  and  Schmitt  (1986).  Items  were  augmented 
slightly  to  fit  the  game  environment.  Sample  items  include  “I  am  motivated  to  learn  the 
information/skills  emphasized  in  the  America’s  Army  game”  and  “I  plan  to  exert  a  lot  of  mental 
effort  to  do  well  in  the  multi-player  missions  of  this  game.”  Possible  responses  ranged  from  1 
(strongly  disagree)  to  5  (strongly  agree).  The  coefficient  alpha  for  this  scale  was  .87.  All  items 
of  this  scale  are  provided  in  Appendix  A. 

Measures  Completed  After  Training 

Metacognitive  activity.  Metacognitive  activity  was  assessed  using  an  eight-item  scale 
adapted  from  Schmidt  and  Ford  (2003).  Items  were  augmented  slightly  to  fit  the  game 
environment.  Sample  items  include  “When  I  practiced  a  new  skill  presented  in  the  game,  I 
monitored  how  well  I  was  learning  its  requirements”  and  “I  thought  about  how  well  my  tactics 
for  playing  the  game  were  working.”  Possible  responses  ranged  from  1  (strongly  disagree)  to  5 
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(strongly  agree).  The  coeffieient  alpha  for  this  scale  was  .89.  All  items  of  this  scale  are 
provided  in  Appendix  A. 

Time  on  task.  Participants  were  asked  to  indicate  the  total  number  of  hours  spent  playing 
the  game  during  the  four  days  allotted  for  this  training  exercise.  We  believe  this  reflects  a 
trainee’s  motivation  to  continue  training,  as  this  videogame-based  training  represents  a  self- 
regulated  voluntary  training  environment. 

Satisfaction  with  training.  Satisfaction  with  the  training  experience  was  assessed  using  a 
three-item  scale  modified  from  Orvis  et  al.  (2005).  An  example  item  is  “I  was  satisfied  with  the 
experience  of  using  the  America’s  Army  game.”  Possible  responses  ranged  from  1  (strongly 
disagree)  to  5  (strongly  agree).  The  coefficient  alpha  for  this  scale  was  .90.  All  items  of  this 
scale  are  provided  in  Appendix  A. 

Ease  in  using  game  interface.  Ease  in  using  the  game  interface  was  assessed  with  three 
items  from  Orvis  et  al.  (2005).  The  items  include  “How  easy/difficult  was  it  to  learn  how  to  use 
America’s  Anny  game?,”  “How  easy/difficult  was  it  to  use  the  menu  system?”  (1  =  very  easy  to 
5  =  very  difficult),  and  “How  comfortable  did  you  feel  using  tbe  system?”  (1  =  very  comfortable 
to  5  =  very  uncomfortable).'  The  coefficient  alpha  for  this  scale  was  .83. 

Team  cohesion.  A  trainee’s  perception  of  the  cohesion  among  his/her  team  members  was 
assessed  using  a  nine-item  scale  adapted  from  Craig  and  Kelly  (1999).  Items  were  augmented 
slightly  to  fit  the  game  environment.  Sample  items  include  “To  what  extent  was  your  team 
engaged  in  the  multi-player  missions  of  the  America’s  Army  game?”  and  “To  what  extent  did 
members  of  your  team  like  being  a  part  of  this  team?”  Because  team  membership  varied  across 
missions  played,  trainees  were  asked  to  respond  to  these  items  with  respect  to  the  most 
successful  team  in  which  they  were  a  team  member.  Possible  responses  ranged  from  1  (not  at 
all)  to  5  (great  extent).  The  coefficient  alpha  for  this  scale  was  .92.  All  items  of  this  scale  are 
provided  in  Appendix  A. 

Training  performance.  Training  game  performance  was  operationalized  at  the  team  level 
as  the  proportion  of  multi-player  missions  the  trainee  won  out  of  the  total  number  of  missions 
completed.  Cadets  were  asked  to  indicate  the  total  number  of  multi-player  missions  completed, 
as  well  as  the  number  of  missions  they  won  and  lost. 

Perceived  transfer  of  videogaming  skills.  To  enhance  our  understanding  of  the  influence 
of  videogame  experience/practice,  cadets  were  posed  a  few  additional  questions.  First,  cadets 
were  asked  to  address  how  helpful  their  previous  overall  videogame  experience  was  in  preparing 
them  to  perform  the  five  skills  used  during  the  multi-player  section  of  America 's  Army.  The 
five  specific  skills  included  a)  using  keyboard  and  mouse  to  control  character,  b)  using  the 
weapons,  c)  interacting  with  team  members,  d)  using  appropriate  combat  tactics,  and  e)  keeping 
up  with  the  pace  of  the  game.  Possible  responses  ranged  from  1  (not  helpful  at  all)  to  5  (very 
helpful).  Second,  cadets  were  asked  to  address  how  helpful  the  single-player  sections  of 
America 's  Army  were  in  preparing  them  to  perform  these  five  different  skills. 


'  These  items  were  reverse  coded  for  the  statistical  analyses  so  that  higher  values  reflected  greater  ease  in  using  the 
interface. 
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RESULTS 


Descriptive  Statistics 

Means,  standard  deviations  and  intercorrelations  of  the  variables  are  displayed  in  Table  1. 
Videogame  Experience 

Overall  videogame  experience.  Results  indicate  that  there  is  a  wide  range  of  prior 
videogame  experience  across  the  participants  in  this  sample,  with  1 2%  of  cadets  reporting  they 
had  no  experience  playing  videogames  in  the  prior  year  and  48%  reporting  they  had  limited 
videogame  experience.  Prior  overall  videogame  experience  significantly  predicted  pretraining 
motivation  {r  =  .7)2,  p  <  .01)  and  most  of  the  examined  learner  choices/outcomes.  Specifically, 
frequency  in  playing  videogames  was  positively  related  to  metacognitive  strategies  utilized 
during  training  {r  =  .27,  p  <  .05),  time  on  task  {r  =  .32,  p  <  .05),  satisfaction  with  the  training 
game  experience  (r  =  .23, p  <  .05),  ease  in  using  the  game  interface  (r  =  .25, p  <  .05),  and 
training  performance  {r  =  .26,  p  <  .05).  The  relationship  between  videogame  experienee  and 
cohesion  with  one’s  team  members  approached  significance  (r  =  .20,  p  <  .10).  Figure  1  displays 
the  relationships  between  overall  videogame  experience  and  the  learner  choices/outcomes,  with 
videogame  experience  dichotomized  at  the  mean  (M=  2.13).  The  error  bars  in  Figure  1 
represent  standard  error  of  the  mean. 


Learner  Choices  and  Outcomes 

Figure  I.  Relationships  between  overall  videogame  experience  and  learner  choices/outcomes 
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Table  1.  Means,  Standard  Deviations,  and  Intercorrelations  of  Variables 


Variable 

N 

M 

SD 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10^^ 

1 .  Overall  gaming 

364 

2.13 

1.43 

- 

experience 

2.  Relevant  gaming 

364 

1.66 

1.24 

.69** 

- 

experience 

3.  First-person- 

363 

2.51 

1.70 

.68** 

.90** 

- 

perspective 

4.  Simulators 

363 

.75 

1.14 

.24** 

.37** 

.35** 

- 

5.  Online 

361 

.80 

1.35 

.39** 

.36** 

.36** 

.24** 

— 

multi-player 

6.  Sports/Racing 

364 

1.95 

1.73 

.39** 

.48** 

.51*’ 

.24** 

.22** 

- 

7.  Command/ 

362 

1.92 

1.87 

.51** 

.58** 

.54** 

.30** 

.44** 

.24** 

Strategy 

8.  Fighting 

362 

1.01 

1.30 

.38** 

.36** 

.36** 

.28** 

.34** 

.39** 

.33“ 

- 

9.  Life/Business 

362 

.83 

1.11 

.26** 

.29** 

.28** 

.22** 

.41** 

.37** 

.35** 

.27“ 

— 

Simulations 

10.  Fantasy 

359 

.91 

1.39 

.34** 

.28** 

.31** 

.32** 

.37** 

.27** 

.40“ 

.58“ 

.35“ 

- 

1 1 .  Puzzles 

363 

1.69 

1.54 

.14** 

.21** 

.25** 

.16** 

.10' 

.25" 

.21“ 

.25“ 

.29“ 

.22“ 

12.  America’s  Army 

364 

.82 

1.21 

.44** 

.79** 

.43** 

.27** 

.23** 

.26“ 

.42“ 

.22“ 

.21“ 

.15“ 

13.  Videogame 

364 

3.78 

1.14 

.59** 

.60** 

.65** 

.24** 

.30** 

.37“ 

.51“ 

* 

• 

OO 

.22 

.32-^ 

self-efficacy 

w 

14.  Learning  goal 

364 

3.82 

.56 

.04 

.18** 

.17** 

.07 

.00 

.02 

.11* 

.00 

-.02 

.02 

orientation 

15.  Performance 

364 

2.72 

.71 

-.01 

-.04 

-.06 

-.05 

-.01 

.03 

-.07 

-.05 

.02 

-.09' 

Avoid  GO 

16.  Performance 

364 

3.34 

.68 

.08 

.14** 

.11* 

.08 

.02 

.16“ 

-.01 

.09 

.09' 

.05 

Prove  GO 

17.  Pretraining 

362 

3.61 

.77 

.32** 

.33** 

.29** 

.15** 

.17“ 

.14“ 

.22“ 

.13* 

.12* 

.10* 

motivation 

18.  Metacognitive 

80 

3.46 

.59 

.27* 

.39** 

.39** 

.21' 

.20' 

.23* 

.30“ 

.17 

.12 

.27* 

activity 

19.  Time  on  task 

54 

2.11 

1.68 

.32* 

.32* 

.26' 

.28* 

-.01 

-.29* 

.14 

-.26' 

-.09 

.07 

20.  Training 

80 

3.74 

.64 

.23* 

.42** 

.39** 

.38** 

.16 

.11 

.28* 

.14 

.17 

.19'  . 

satisfaction 

21 .  Ease  in  using 

80 

3.57 

.78 

.25* 

.46** 

.48** 

.30** 

.17 

.22* 

.40** 

.05 

.13 

.17 

interface 

22.  Team  cohesion 

75 

3.27 

.71 

.20' 

.41** 

.33** 

.15 

.20' 

.12 

.38“ 

.04 

.08 

.30* 

23.  Training 

65 

37.1 

38.4 

.26* 

.33** 

.24' 

.15 

.22' 

.17 

.27* 

.25* 

.09 

.29* 

performance 


Note.  All  variables  were  measured  on  a  1  -5  scale  except  overall  and  relevant  gaming  experience  (measured  on  a 
0-5  scale),  time  on  task  (measured  in  hours)  and  training  performance  (measured  as  percent  of  missions  won  out 
of  total  completed).  ^p<.10.  *p<.05.  **/><  .01  (two-tailed). 
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Table  1.  (continued) 


Variable 


11  12  13  14  15  16  17  18  19  20  21  22 


1 .  Overall  gaming 
experience 

2.  Relevant  gaming 
experience 

3.  First-person- 
perspective 

4.  Simulators 

5.  Online 
multi-player 

6.  Sports/Racing 

7.  Command/ 
Strategy 

8.  Fighting 

9.  Life/Business 
Simulations 

10.  Fantasy 

1 1 .  Puzzles 


12.  America’s  Army  .06 

13.  Videogame  .11* 

self-efficacy 

14.  Learning  goal  .06 

orientation 

15.  Performance  -.12 

Avoid  GO 

16.  Performance  .04 

Prove  GO 

17.  Pretraining  .11* 

motivation 

18.  Metacognitive  .06 

activity 

19.  Time  on  task  -.02 

20.  Training  .17 

satisfaction 

21 .  Ease  in  using  .20^ 

interface 

22.  Team  cohesion  .10 

23.  Training  .20 

performance 


.32 


.13* 

.22 

- 

.00 

-.02 

-.11* 

- 

.12* 

.14** 

.40" 

.40" 

- 

.27** 

.38" 

.41" 

.15" 

.32" 

.26* 

.18 

.47" 

-.05 

.18 

.30* 

.26^ 

.03 

-.23^ 

-.22 

.32** 

.35" 

.44" 

-.18 

.27* 

.27* 

.45** 

.16 

-.26* 

.03 

.38** 

.11 

.15 

.06 

.09 

.36** 

.29* 

-.03 

.05 

.07 

.33" 

- 

.07 

.12 

-w 

.30" 

.63" 

.25^ 

- 

.16 

.14 

.26* 

.35" 

- 

.15 

.57" 

.26^ 

.36" 

.57' 

.04 

.11 

.17 

.06 

.11 

.17 
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Specific  videogame  experience.  The  role  of  prior  experience  with  specific  game  types  on 
learner  choices/outcomes  was  examined.  Seven  regression  analyses  were  conducted,  where  each 
of  the  seven  criteria  (pretraining  motivation,  metacognitive  activity,  time  on  task,  training 
satisfaction,  ease  in  using  the  game  interface,  team  cohesion,  and  training  performance)  were 
regressed  onto  the  specific  types  of  game  experiences  (see  Table  2).  Generally,  results  indicate 
that  prior  experiences  relevant  to  the  videogame  used  in  the  training  were  significant  predictors. 
Specifically,  previous  experience  with  ihe  America 's  Army  game  was  a  unique  predictor  for  four 
of  the  examined  criteria  (i.e.,  pretraining  motivation,  time  on  task,  team  cohesion,  and  training 
performance).  Prior  experience  with  other  first-person-perspective  games  was  also  a  significant 
predictor  of  most  of  the  criteria,  including  pretraining  motivation,  time  on  task,  training 
satisfaction,  and  ease  in  using  the  game  interface;  it  approached  significance  for  metacognitive 
activity.  Experience  using  other  types  of  specific  games  which  did  not  share  several  similar 
characteristics  to  the  current  training  game,  such  as  puzzles/card  games/board  games, 
life/business  simulations,  and  massively  multi-player  online  games,  were  not  predictive  of  any  of 
the  examined  criteria.  Prior  gaming  experience  with  flight  simulators  was  found  to  prediet  time 
on  task  and  training  satisfaction.  Prior  experience  with  fighting  games  was  found  to  negatively 
predict  several  of  the  learner  choices/outcomes  (i.e.,  time  on  task,  ease  in  using  the  game 
interface,  and  team  cohesion).  In  summary,  consistent  with  the  Year  1  findings,  these  results 
suggest  that  only  experience  with  relevant  videogames  positively  predicts  subsequent  learner 
ehoiees/outcomes  in  a  game-based  training  environment.  The  game  experiences  most  relevant  to 
the  training  game  used  in  this  research  are  prior  experience  with  America ’s  Army  and  other  first- 
person-perspective  games. 

Perceived  transfer  of  videogaming  skills.  As  a  means  for  further  enhancing  our 
understanding  of  the  role  of  prior  videogame  experience/practice  with  games,  participants  were 
asked  a  few  additional  questions.  Trainees  were  asked  to  report  how  helpful  their  previous 
overall  videogame  experience  was  in  preparing  them  to  perform  five  different  skills  used  during 
the  multi-player  section  of  America 's  Army  (e.g.,  using  keyboard  and  mouse  to  control  character, 
using  the  weapons,  using  appropriate  combat  tactics,  interacting  with  team  members,  and 
keeping  up  with  the  pace  of  the  game).  Trainees  reported  that  their  prior  gaming  experienee  was 
helpful  in  preparing  for  the  multi-player  section.  On  a  scale  ranging  from  not  helpful  at  all  (1)  to 
very  helpful  (5),  the  mean  helpfulness  rating  across  the  five  skills  ranged  from  3.56  to  4.05. 

(Table  3  provides  the  mean  helpfulness  rating/standard  deviation  for  each  skill.)  Trainees  were 
also  asked  to  report  how  helpful  the  single-player  sections  oi  America 's  Army  were  in  preparing 
them  to  perform  the  skills  used  during  the  multi-player  section.  On  average,  trainees  reported 
that  the  single-player  section  was  moderately  helpful  in  preparing  for  the  multi-player  seetion. 

The  mean  helpfulness  rating  across  the  five  skills  ranged  from  3.12  to  3.65.  This  suggests  that 
trainees  perceive  that  their  prior  videogame  experience  and/or  practice  with  the  particular  game 
used  for  training  was  beneficial  to  their  performance  in  the  training  game. 
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Table  3.  Perceived  Helpfulness  of  Prior  Videogame  Experience/Practice 


Prior  Videogame 

Sinele-Plaver  Section 

Experience 

America 's  Army 

Skills 

M 

SD 

M 

SD 

Using  keyboard/mouse  to  control  character 

4.03 

1.01 

3.58 

.90 

Using  the  weapons 

4.05 

.96 

3.65 

.86 

Using  appropriate  combat  tactics 

3.56 

1.09 

3.23 

1.01 

Interacting  with  team  members 

3.58 

1.00 

3.12 

1.05 

Keeping  up  with  pace  of  the  game 

3.82 

.99 

3.30 

1.00 

Videogame  Self-Efficacy  and  Goal  Orientation 

The  impact  of  trainees’  self-efficacy  and  goal  orientation  on  pretraining  motivation,  as 
well  as  on  several  learner  choices  and  outcomes,  was  assessed  using  hierarchical  regression.  For 
each  criterion,  a  hierarchical  regression  analysis  was  conducted  with  prior  relevant  videogame 
experience  in  Step  1,  videogame  self-efficacy  in  Step  2,  and  the  three  types  of  goal  orientation  in 
Step  3.  Prior  relevant  videogame  experience  (i.e.,  a  mean  composite  of  first-person-perspective 
and  America 's  Army  experience)  was  entered  in  the  first  step  because  our  Year  1  findings 
demonstrated  that  this  trainee  characteristic  significantly  predicts  learner  choices/outcomes  of 
videogame-based  training  environments  (Orvis  et  al.,  2005).  We  entered  videogame  self- 
efficacy  in  Step  2  because  in  Year  1  a  similar  type  of  self-efficacy,  computer  self-efficacy, 
significantly  predicted  the  learner  outcomes. 

The  trainee  individual  characteristics  accounted  for  32%  of  the  variance  in  pretraining 
motivation  (p  <  .01).  Specifically,  prior  relevant  videogame  experience  .1 1,/>  <  .01), 
videogame  self-efficacy  (A/?^=  .05, p  <  .01),  and  goal  orientation  {IS.R^=  .15,/?  <  .01) 
significantly  predicted  trainees’  pretraining  motivation.  Examination  of  the  individual  beta 
weights  indicates  support  for  Learning  goal  orientation  (fi  =  .32,  p  <  .01)  and  Performance  Avoid 
goal  orientation  (/?  =  .16, /?  <  .01).  Performance  Prove  goal  orientation  did  not  significantly 
predict  pretraining  motivation. 

The  trainee  individual  characteristics,  as  a  set,  also  significantly  predicted  the  learner 

2  2 

choices/outcomes  of  metacognitive  activity  {R  =  .33, p  <  .01),  time  on  task  {R  =  .20, p  <  .05), 
satisfaction  with  the  game-based  training  {R  =  .36,  p  <  .01),  perceived  ease  in  using  the  game 
interface  {I^  =  .32,  p  <  .01),  and  perceived  team  cohesion  {Pr=  .21,/?  <  .01).  Supporting  our 
Year  1  findings,  the  results  indicate  that  prior  relevant  videogame  experience  explained  a 
substantial  amount  of  variance  in  all  of  the  learner  choices/outcomes:  metacognitive  activity 
=  .16,/?  <  .01),  time  on  task  {R^  =  .!!,/?<  .05),  training  satisfaction  {I^=  .18,/?  <  .01),  ease  in 
using  the  game  interface  {I^=  .21,/?  <  .01),  team  cohesion  {I^=  .17,/?  <  .01),  and  training 
performance  (7?^=  .11,/?  <  .01).  Trainees’  self-efficacy  for  playing  videogames  explained  a 
significant  amount  of  variance  above  and  beyond  videogame  experience  in  ease  in  using  the 
game  interface  (A/?^  =  .05,  p  <  .05). 
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Finally,  trainee  goal  orientation  explained  a  small  to  substantial  increment  of  variance  in 
several  of  the  learner  choices/outcomes,  including  metacognitive  activity  =  .  1 7,  /?  <  .01 ), 

training  satisfaction  (A/?^  =  .  1 7,  /?  <  .01 ),  and  ease  in  using  the  game  interface  (A/?^  =  .06,  p<.  1 0). 
Examination  of  the  individual  beta  weights  indicates  that  Learning  goal  orientation  positively 
predicted  trainee  engagement  in  metacognitive  activity  during  training  ifi  =  .44,  p  <  .01)  and 
satisfaction  with  the  game-based  training  {fi  =  26,  p  <  .05).  Performance  Avoid  goal  orientation 
negatively  predicted  one’s  training  satisfaction  =  -.22,  p  <  .05)  and  perceived  ease  in  using  the 
game  interface  {Ji  =  -21,  p  <  .05).  Performance  Prove  goal  orientation,  as  expected,  was 
generally  not  predictive  of  the  learner  choices/outcomes.  However,  the  beta  weight  for 
Performance  Prove  goal  orientation  approached  significance  for  training  satisfaction  {fi-  2\  ,P< 
.  1 0).  Results  are  presented  in  Table  4. 


DISCUSSION 

The  purpose  of  this  paper  was  to  investigate  the  influence  of  trainee  individual  attributes 
on  trainee  motivation,  learner  choices,  and  learning  outcomes  of  videogame-based  training 
environments.  This  paper  extends  existing  research  by  demonstrating  the  importance  of  a 
trainee’s  prior  videogame  experience,  videogame  self-efficacy,  and  goal  orientation  for  the 
prediction  and  explanation  of  these  training  criteria.  Our  findings  in  relation  to  each  trainee 
characteristic  are  discussed  below,  along  with  practical  implications  of  this  research  and 
suggestions  for  future  research. 

With  regard  to  prior  videogame  experience,  consistent  with  our  Year  1  findings,  a 
videogame  genre-specific  effect  was  demonstrated  in  that,  in  general,  only  specific  prior  game 
experiences  that  share  similar  characteristics  with  the  current  training  game  positively  predicted 
the  learner  choices/outcomes.  Trainees  with  greater  experience  in  playing  videogames  relevant 
to  the  current  training  game  environment  (i.e.,  America ’s  Army  and  other  first-person-perspective 
games)  were  more  motivated  to  train  using  the  training  videogame,  more  comfortable  and 
satisfied  using  the  training  videogame,  and  more  cohesive  with  their  teammates  during  the 
collaborative  components  of  the  training  game.  More  experienced  individuals  also  made  more 
effective  learner  choices  during  training  than  those  with  less  experience.  Specifically,  they 
thought  more  about  how  well  they  were  learning  the  information/skills  presented  during  the 
training  game  and  strategies  they  could  use  to  improve  their  level  of  learning.  They  also  spent 
more  time  engaging  in  the  training  game  than  individuals  with  lower  levels  of  prior  relevant 
game  experience.  In  contrast,  prior  experiences  with  specific  videogames  that  do  not  share 
similar  characteristics  with  the  current  training  game  were  generally  not  positively  related  to  the 
examined  learner  choices/outcomes. 

It  is  worth  noting  that  prior  gaming  experience  with  flight  simulators  (e.g..  Falcon, 
Microsoft  Flight  Simulator,  Lock  On:  Modern  Air)  was  found  to  positively  predict  a  few  of  the 
learner  outcomes,  including  training  satisfaction  and  time  on  task.  In  Year  1  of  this  research, 
experience  with  this  game  type  predicted  time  on  task.  It  may  be  that  flight  simulation  games  are 
more  closely  associated  to  training  well-defined  skills  (versus  solely  providing  entertainment)  as 
compared  to  other  types  of  videogames.  Another  explanation  for  this  relationship  may  be  that 
America 's  Army  and  simulations  share  common  game  features  or  characteristics  (e.g.,  first 
person  perspective  in  a  virtual  3D  world  and  pacing  of  game  actions/events)  that  are  critical  to 
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game  success.  Future  research  should  investigate  the  relationship  among  these  two  game  t3^es 
and  the  features  they  share  with  one  another. 

Next,  self-efficacy  for  playing  videogames  was  examined  for  its  unique  contribution  to 
the  prediction  of  the  learner  choices/outcomes,  after  controlling  for  prior  relevant  videogame 
experience.  Trainees’  self-efficacy  for  playing  videogames  explained  a  significant  amount  of 
variance  above  and  beyond  videogame  experience  in  pretraining  motivation  and  ease  in  using  the 
game  interface.  This  indicates  that  trainees  with  greater  confidence  in  playing  videogames  are 
more  motivated  and  comfortable  training  in  game-based  environments  than  trainees  with  lower 
videogame  self-efficacy  beliefs.  Note  that  when  self-efficacy  was  examined  independently  as  a 
predictor  (i.e.,  separately  from  prior  experience),  this  trainee  characteristic  also  significantly 
predicted  satisfaction  with  the  training  experience  and  training  performance,  and  approached 
significance  for  time  on  task.  Thus,  videogame  self-efficacy  is  influential  on  several  trainee 
learner  ehoices/outeomes  of  game-based  training  environments;  however,  it  does  not  add 
substantially  to  the  prediction  of  most  of  these  criteria  beyond  trainees’  previous  experience 
levels. 


Trainees’  goal  orientation  also  had  a  unique  impact  on  the  learner  choices/outcomes.  In 
other  words,  knowledge  of  individuals’  goal  orientations  provided  unique  information,  beyond 
their  prior  experience  and  self-efficacy  expectations,  to  the  prediction  of  their  motivation  to  train 
and  reactions  to/experiences  with  the  videogame-based  training  program. 

As  expected.  Learning  goal  orientation  was  positively  associated  with  pretraining 
motivation,  metacognitive  activity,  and  the  affect-based  learning  outcome  of  training 
satisfaction.  These  findings  suggest  that  trainees  who  approach  instructional  environments  with 
the  purpose  of  mastering  new  knowledge/skills  are  more  motivated  to  train  and  more  satisfied 
learning  in  a  game-based  training  environment,  as  compared  to  trainees  with  a  low  Learning  goal 
orientation.  Further,  learning-oriented  trainees  made  more  effective  learner  choices  during  the 
training.  They  were  more  active  in  monitoring  their  learning  progress  and  in  implementing  new 
strategies  to  address  any  difficulties  they  were  experiencing  in  learning  the  content  of  the 
training  game.  These  findings  are  consistent  with  past  research  indicating  that  Learning  goal 
orientation  positively  impacts  learner  choices/outcomes  in  other  technology-delivered  training 
environments,  such  as  e-leaming  (e.g..  Brown,  2001;  Ford  et  ah,  1998;  Schmidt  &  Ford,  2003). 

Performance  Avoid  goal  orientation  generally  exhibited  a  negative  impact  on  the  learner 
choices/outcomes,  including  training  satisfaction,  perceived  ease  in  using  the  game  interface,  and 
time  on  task.  These  findings  suggest  that  trainees  who  seek  to  avoid  receiving  negative 
evaluations  from  others,  or  demonstrating  low  ability/skill  levels,  are  less  comfortable  and 
satisfied  learning  in  a  game-based  training  environment.  Further,  they  spend  less  time  engaging 
in  the  training  game  as  compared  to  trainees  with  a  low  Performance  Avoid  goal  orientation. 
Intuitively,  these  findings  make  sense  as  videogame-based  training  environments  (such  as  the 
training  game  used  in  the  present  research)  often  require  trainees  to  work  collaboratively  in  order 
to  be  successful  in  the  game  and  to  learn  the  instructional  content.  This  high  level  of 
collaboration  during  training  may  increase  a  high  Performanee  Avoid  learner’s  perceptions 
concerning  his/her  likelihood  of  demonstrating  low  ability/skill  levels;  and,  therefore,  result  in 
negative  training-related  cognitions  and  behaviors. 
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Contrary  to  our  expectations  and  those  of  prior  work,  Performance  Avoid  goal  orientation 
was  found  to  be  positively  related  to  pretraining  motivation.  To  investigate  a  possible  reason  for 
this  unexpected  finding,  we  revisited  our  measurement  of  pretraining  motivation.  Several  of  the 
items  reflected  a  trainee’s  motivation  to  learn  the  training  content,  while  a  few  items  reflected 
more  of  a  motivation  to  succeed/perform  well  in  the  training  (e.g.,  “I  am  motivated  to  learn  the 
information/skills  emphasized  in  the  America’s  Army  game”  versus  “I  will  feel  upset  if  I 
perform  poorly  in  the  multi-player  missions  of  this  game”).  Therefore,  we  divided  the  items  into 
two  subscales  reflecting  these  two  aspects  of  training  motivation.  Correlations  between  these 
subscales  and  Performance  Avoid  goal  orientation  suggest  that  high  Performance  Avoid  learners 
were  motivated  to  perform  well  in  the  training  game  (r  =  .18,/?  <  .01);  however,  they  were  not 
motivated  to  learn  from  the  training  (r  =  .07,  ns).  In  other  words,  a  possible  reason  underlying 
their  high  training  motivation  was  to  avoid  demonstrating  low  ability/skill  levels  or  receiving 
negative  evaluations  concerning  their  capabilities,  rather  than  to  work  hard  to  actually  master  the 
knowledge/skills  taught  in  the  game.  In  contrast,  high  learning  goal-oriented  trainees  likely 
valued  learning  new  knowledge/skills  (r=  .31,  p<  .01 ).  Further,  they  may  have  perceived  the 
items  reflecting  high  training  performance  as  a  reflection  of  their  substantive  knowledge/skill 
development  (r  =  .31,  p  <  .01). 

Finally,  in  general.  Performance  Prove  goal  orientation  did  not  significantly  predict  the 
learner  choices/outcomes  in  the  hierarchical  regression  analyses.  Performance  Prove  goal 
orientation  did  approach  significance  in  its  prediction  of  trainees’  satisfaction  with  the  training 
experience.  This  finding  suggests  that  trainees  who  seek  to  outperform  others  and  demonstrate 
high  ability/skill  levels  are  more  satisfied  engaging  in  a  game-based  training  environment  than 
those  with  a  low  Perfonnance  Prove  goal  orientation.  A  possible  explanation  for  this  unexpected 
finding  may  be  that  these  individuals  perceived  the  highly  collaborative  nature  of  the  game- 
based  training  as  an  opportunity  to  demonstrate  their  competence  or  outshine  other  trainees; 
therefore,  they  enjoy  this  type  of  learning  environment.  As  U.S.  Military  Academy  cadets  may 
not  be  representative  of  the  general  population,  future  research  should  investigate  if  this 
relationship  is  replicated  with  a  different  sample  of  trainees. 

Implications  and  Directions  for  Future  Research 

This  research  suggests  that  the  attributes  trainees  bring  to  the  training  environment  are 
important  variables  to  consider  when  implementing  game-based  training.  A  trainee’s  prior 
experience,  self-efficacy,  and  goal  orientation  beliefs  contributed  value  added  in  terms  of 
understanding  when  game-based  training  should  succeed  in  enhancing  trainee  motivation  and 
other  important  learner  choices/outcomes.  The  good  news  is  that  these  attributes  are  relatively 
malleable  trainee  characteristics  that  can  be  influenced  by  instructors  or  game  developers.  Based 
on  the  present  research  findings,  we  provide  some  specific  recommendations  for  instructors 
utilizing  game-based  training  and  training  game  developers. 

First,  we  suggest  that  instructors  assess  trainees’  prior  game  experiences.  By  assessing 
the  amount  of  relevant  previous  gaming  experiences  trainees  possess,  instructors  will  be  able  to 
identify  those  who  lack  the  prerequisite  game  experience.  In  turn,  instructors  can  then  provide 
these  trainees  with  targeted  opportunities  to  gain  such  beneficial  experiences  prior  to  training. 

For  example,  if  learners  are  to  engage  in  a  first-person-perspective  game-based  training  program 
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and  some  learners  have  little  prior  experience  with  this  type  of  videogame,  then  the  instructor 
would  know  to  give  them  ample  practice  time  before  the  learning  segment  of  the  training  (i.e., 
when  learners  are  acquiring  the  new  knowledge/skills  taught  in  the  game).  To  facilitate 
instructors  in  providing  the  appropriate  amount  of  preparatory  practice  for  a  given  learner’s 
needs,  training  game  developers  should  incorporate  a  feature  within  the  game  that  enables  the 
instructor  to  select  the  desired  amount  and  content  of  trainee  orientation  and  practice. 

It  may  be  assumed  that  most  junior  Soldiers  who  grew  up  in  the  digital  age  would  have  a 
great  deal  of  experience  with  videogames;  and,  therefore,  additional  orientation  and  practice  with 
videogames  would  be  unnecessary.  This  assumption  does  not  seem  warranted  given  the 
experience  levels  of  the  participants  sampled.  In  the  current  sample,  12%  of  first-year  U.  S. 
Military  Academy  cadets  (mean  age  =  18.89  years)  reported  they  had  no  prior  videogame 
experience  of  any  kind  (i.e.,  the  overall  videogame  experience  measure)  and  48%  reported  they 
had  limited  experience.  These  findings  parallel  our  Year  1  results,  in  which  17%  of  cadets 
reported  they  had  no  experience  playing  videogames  and  44%  reported  they  had  limited 
videogame  experience.  Further,  16%  of  cadets  reported  they  have  no  experience  playing  first- 
person-perspective  videogames  or  America 's  Army  in  the  past  year  and  65%  reporting  they  have 
limited  experience  with  this  type  of  videogame.  Given  the  number  of  cadets  with  little  to  no 
experience,  providing  an  orientation  or  additional  practice  with  relevant  games  would  likely  be 
valuable  whenever  implementing  a  game-based  training  program.  Doing  so  may  improve  a  host 
of  learner  choices/outcomes  such  as  trainee  motivation,  metacognitive  activity,  length  of  time 
devoted  to  training,  and  training  satisfaction. 

When  instructors  are  providing  additional  practice  opportunities  with  a  relevant 
videogame,  it  is  also  likely  that  trainees  will  feel  more  confident  in  their  capability  to 
successfully  learn  in  a  training  environment  which  incorporates  a  comparable  game.  One  way  to 
enhance  self-efficacy  is  to  initially  provide  relatively  easy  practice  sessions.  Then,  when 
learners  are  successful  at  these  practice  sessions,  provide  positive  feedback  and  encouragement. 
Such  feedback  could  be  provided  by  the  instructor  or  could  be  built  into  the  videogame  content 
and  delivered  during  game  play.  These  suggestions  are  supported  by  Bandura  (1977)  who 
proposed  that  obtaining  experiences  resulting  in  successful  performance  or  receiving  feedback 
on  one’s  capabilities  are  two  ways  in  which  to  develop  positive  self-efficacy  beliefs. 

With  respect  to  goal  orientation,  while  this  construct  is  often  characterized  as  a  fairly 
stable  personal  trait,  the  goal  orientation  literature  does  suggest  that  an  individual’s  goal 
orientation  can  be  shaped  by  situational  factors  (e.g.,  Button  et  al.,  1996;  Kozlowski  et  al.,  2001; 
Schmidt  &  Ford,  2003;  Weissbein  &  Ford,  2002).  For  instance,  Weissbein  and  Ford  (2002) 
found  that  a  pretraining  intervention  was  successful  at  influencing  trainees’  attributions,  such 
that  trainees  adopted  a  more  learning  goal-oriented  perspective  while  engaging  in  a  training 
program.  Accordingly,  we  recommend  that  instructors  implementing  a  game-based  training 
program  should  emphasize  the  perspective  of  learning  and  acquiring  new  skills  during  the  game 
versus  the  perspective  of  striving  reach  a  high  score  or  to  be  the  “best”  at  the  game. 

The  results  of  the  present  research  suggest  several  interesting  points  worth  further 
consideration  and  additional  research.  First,  we  realize  that  the  primary  limitation  of  the  present 
research  was  that  we  were  unable  to  measure  knowledge  or  skill-based  learning  outcomes. 


21 


While  the  learner  choices/outcomes  examined  in  this  research  have  been  associated  with  learning 
in  other  training  contexts,  future  research  is  needed  to  enhance  our  understanding  of  the  role  of 
trainee  characteristics  in  predicting  these  more  distal  criteria  in  game-based  learning 
environments. 

The  present  research  did  attempt  to  measure  performance  while  completing  the  training, 
as  prior  research  has  shown  that  a  learner’s  training  performance  is  positively  related  to 
knowledge/skill  transfer  (Ford  et  ah,  1998).  We  measured  training  performance  as  the  self- 
reported  proportion  of  collaborative  missions  the  learner  won  out  of  the  total  number  of  missions 
completed.  It  is  important  to  note  that  we  were  unable  to  capture  and  assess  performance  of  an 
individual  learner.  Instead,  this  measure  captured  team-level  performance  during  the  missions 
played.  While  an  individual  learner’s  level  of  performance  may  directly  contribute  to  (or 
correlate  with)  his/her  team’s  performance,  this  assumption  may  not  necessarily  be  the  case. 

This  may  account  for  why  we  found  that  the  trainee  characteristics  accounted  for  a  smaller 
proportion  of  variance  in  training  performance,  as  compared  to  the  other  examined  learner 
choices/outcomes,  which  could  be  assessed  at  the  individual-level.  We  suggest  future  research 
incorporate  individual-level  assessments  of  game  performance  and/or  of  the  learner’s  mastery 
level  of  knowledge  and  skills  being  taught  during  the  training  game. 

Another  avenue  for  future  research  is  to  examine  how  trainee  characteristics  and  common 
features  or  attributes  incorporated  in  a  training  game  may  interact  to  influence  trainees’  choices 
made  during  the  training  and  learning  outcomes.  Prior  research  has  demonstrated  such  attribute- 
treatment  interactions  within  other  types  of  training  environments.  For  example,  trainee 
characteristics,  such  as  cognitive  ability  and  anxiety,  have  been  found  to  interact  with 
features/attributes  (e.g.,  structure)  of  classroom-based  instructional  environments  to  influence  a 
learner’s  level  of  learning  and  retention  (see  Campbell  &  Kuncel,  2001 ;  Rigney,  Munro,  & 

Crook,  1979;  Snow,  1989;  Snow  &  Lohman,  1984).  We  suggest  future  research  investigate  such 
possible  interactions  in  a  game-based  learning  environment  and  consider  whether  game  features 
can  be  altered  to  better  accommodate  different  types  of  trainees. 

Summary 

Desktop  simulations  and  gaming  technology  have  captured  the  attention  of  training 
professionals  and  educators  (Garris  et  ah,  2002)  and  PC-based  videogames  are  emerging  as  an 
increasingly  popular  training  tool  in  the  U.  S.  Army  (Herz  &  Macedonia,  2002).  To  date,  little 
research  had  been  directed  towards  identifying  specific  individual  characteristics  of  the  trainee 
that  may  facilitate  or  impede  the  effectiveness  of  videogames  as  training  and  development  tools. 
The  present  research  demonstrates  that  the  attributes  Soldiers  bring  to  the  game-based  training 
environment  may  influence  their  motivation  to  train,  as  well  as  several  other  learner  choices  and 
outcomes  relevant  to  this  type  of  learning  environment.  Specifically,  trainees’  prior  videogame 
experience,  videogame  self-efficacy,  and  goal  orientation  made  significant  differential 
contributions  to  the  prediction  of  these  proximal  training  criteria.  Maximizing  these  proximal 
criteria,  in  tum,  should  impact  Soldiers’  overall  mastery  of  the  knowledge  and  skills  being 
trained  and  ultimately  enhance  their  operational  capabilities. 
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APPENDIX  A 


Goal  orientation  scale 

Learning  GO  (Items  1-5);  Performance  Avoid  GO  (Items  6-9);  Performance  Prove  GO  (Items  10-13) 
Response  options:  1  (strongly  agree)  to  5  (strongly  disagree) 

1 .  lam  willing  to  select  a  challenging  task/assignment  that  I  can  learn  a  lot  from. 

2.  I  often  look  for  opportunities  to  develop  new  skills  and  knowledge. 

3.  I  enjoy  challenging  and  difficult  tasks  at  school  where  I'll  learn  new  skills. 

4.  For  me,  development  of  my  ability  is  important  enough  to  take  risks. 

5.  I  prefer  to  work  in  situations  that  require  a  high  level  of  ability  and  talent. 

6.  I  would  avoid  taking  on  a  new  task  if  there  was  a  chance  that  1  would  appear  rather  incompetent 
to  others. 

7.  Avoiding  poor  performance  is  more  important  to  me  than  learning  a  new  skill. 

8.  I’m  concerned  about  taking  a  task/assignment  at  school  if  my  performance  would  reveal  that  1 
had  low  ability. 

9.  I  prefer  to  avoid  situations  at  school  where  I  might  perform  poorly. 

10.  I’m  concerned  that  I  show  that  I  can  perform  better  than  my  classmates. 

1 1 . 1  try  to  figure  out  what  it  takes  to  prove  my  ability  to  others  at  school. 

12. 1  enjoy  it  when  others  at  school  are  aware  of  how  well  I  am  doing. 

13.1  prefer  to  work  on  tasks/assignments  where  I  can  prove  my  ability  to  others. 

Satisfaction  with  training  scale 

Response  options:  1  (strongly  agree)  to  5  (strongly  disagree) 

1 .  I  was  satisfied  with  the  experience  of  using  the  America’s  Army  game. 

2.  I  was  satisfied  with  the  information/skills  emphasized  in  the  America’s  Army  game. 

3.  Playing  the  America’s  Army  in  this  training  exercise  was  enjoyable  for  me. 

Pretraining  motivation  scale 

Response  options:  1  (strongly  agree)  to  5  (strongly  disagree) 

1 .  lam  motivated  to  learn  the  information/skills  emphasized  in  the  America’s  Army  game. 

2.  I  will  exerted  considerable  effort  to  learn  the  information/skills  presented  in  this  game. 

3.  lam  eager  to  do  well  in  the  America’s  Army  multi-player  missions  I  will  be  playing. 

4.  1  plan  to  exert  a  lot  of  mental  effort  to  do  well  in  the  multi-player  missions  of  this  game. 

5.  1  will  feel  upset  if  I  perform  poorly  in  the  multi-player  missions  of  this  game. 


Continued  on  the  following  page 


Metacognitive  activity  scale 

Response  options:  1  (strongly  agree)  to  5  (strongly  disagree) 

1 .  If  I  got  confused  while  playing,  I  made  sure  I  sorted  it  out  before  moving  on. 

2.  In  order  to  direct  my  efforts  I  thought  about  the  things  I  needed  to  do  to  leam. 

3.  I  tried  to  determine  which  things  I  didn’t  understand  well  and  adjusted  my  learning  strategies 
accordingly. 

4.  When  I  practiced  a  new  skill  presented  in  the  game,  I  monitored  how  well  I  was  learning  its 
requirements. 

5.  I  noticed  where  I  made  mistakes  and  focused  on  improving  those  areas. 

6.  I  tried  to  monitor  closely  the  skills  where  I  needed  to  improve  most. 

7.  1  thought  about  how  well  my  tactics  for  playing  the  game  were  working. 

8.  I  tried  to  change  my  strategies  for  playing  the  game  in  order  to  fit  the  demands  of  the  mission. 

Team  cohesion  scale 

Response  options:  1  (not  at  all)  to  5  (to  a  great  extent) 

1 .  To  what  extent  was  your  team  engaged  in  the  multiplayer  missions  of  the  America’s  Army 
game? 

2.  To  what  extent  did  your  team  enjoy  working  on  the  America’s  Army  game? 

3.  To  what  extent  did  your  team  treat  the  exercise  using  America’s  Army  game  as  meaningful 
and  important? 

4.  To  what  extent  did  you  expect  your  team  to  derive  benefits  from  a  successful  team  performance 
during  America’s  Army  game? 

5.  To  what  extent  did  the  members  of  your  team  like  each  other? 

6.  To  what  extent  was  it  important  that  the  members  of  your  team  got  along  with  one  another? 

7.  To  what  extent  did  the  members  of  your  team  feel  similar  to  one  another? 

8.  To  what  extent  was  it  important  for  members  of  your  team  to  communicate  during  the  mission? 

9.  To  what  extent  did  members  of  your  team  like  being  a  part  of  this  team? 
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